model fail
Studying the Korean Word-Chain Game with RLVR: Mitigating Reward Conflicts via Curriculum Learning
Reinforcement learning with verifiable rewards (RLVR) is a promising approach for training large language models (LLMs) with stronger reasoning abilities. It has also been applied to a variety of logic puzzles. In this work, we study the Korean word-chain game using RLVR. We show that rule-derived rewards can naturally conflict, and demonstrate through experiments that a curriculum-learning scheme mitigates these conflicts. Our findings motivate further studies of puzzle tasks in diverse languages.
3eb65004054f5d21fca4087f5658c727-AuthorFeedback.pdf
Thanks for the insightful and helpful reviews, which will significantly improve our paper. R1, R2, R3 indicate to whom the concern belongs. Ground truth is in red, predictions are in blue, and predicted eye gaze point of the gaze-based model is in green. SVMo bridges global and local context both spatially ( e.g., whole frame vs anchor Other contributions include exhaustive experiments which may be useful for future studies. In (c), the predicted gaze falls on the intersection of 3 objects, slightly closer to the center of the rabbit.
"Why did the Model Fail?": Attributing Model Performance Changes to Distribution Shifts
Zhang, Haoran, Singh, Harvineet, Ghassemi, Marzyeh, Joshi, Shalmali
Machine learning models frequently experience performance drops under distribution shifts. The underlying cause of such shifts may be multiple simultaneous factors such as changes in data quality, differences in specific covariate distributions, or changes in the relationship between label and features. When a model does fail during deployment, attributing performance change to these factors is critical for the model developer to identify the root cause and take mitigating actions. In this work, we introduce the problem of attributing performance differences between environments to distribution shifts in the underlying data generating mechanisms. We formulate the problem as a cooperative game where the players are distributions. We define the value of a set of distributions to be the change in model performance when only this set of distributions has changed between environments, and derive an importance weighting method for computing the value of an arbitrary set of distributions. The contribution of each distribution to the total performance change is then quantified as its Shapley value. We demonstrate the correctness and utility of our method on synthetic, semi-synthetic, and real-world case studies, showing its effectiveness in attributing performance changes to a wide range of distribution shifts.
- North America > United States > Virginia > Arlington County > Arlington (0.04)
- North America > United States > New York (0.04)
- North America > United States > Massachusetts > Middlesex County > Cambridge (0.04)
- (2 more...)
La veille de la cybersécurité
Deep convolutional neural networks (DCNNs) do not view things in the same way that humans do (through configural shape perception), which might be harmful in real-world AI applications. This is according to Professor James Elder, co-author of a York University study recently published in the journal iScience. The study, which conducted by Elder, who holds the York Research Chair in Human and Computer Vision and is Co-Director of York's Centre for AI & Society, and Nicholas Baker, an assistant psychology professor at Loyola College in Chicago and a former VISTA postdoctoral fellow at York, finds that deep learning models fail to capture the configural nature of human shape perception. In order to investigate how the human brain and DCNNs perceive holistic, configural object properties, the research used novel visual stimuli known as "Frankensteins." "Frankensteins are simply objects that have been taken apart and put back together the wrong way around," says Elder. "As a result, they have all the right local features, but in the wrong places."
AI Use Potentially Dangerous "Shortcuts" To Solve Complex Recognition Tasks
The researchers revealed that deep convolutional neural networks were insensitive to configural object properties. Deep convolutional neural networks (DCNNs) do not view things in the same way that humans do (through configural shape perception), which might be harmful in real-world AI applications, according to Professor James Elder, co-author of a York University study recently published in the journal iScience. The study, which conducted by Elder, who holds the York Research Chair in Human and Computer Vision and is Co-Director of York's Centre for AI & Society, and Nicholas Baker, an assistant psychology professor at Loyola College in Chicago and a former VISTA postdoctoral fellow at York, finds that deep learning models fail to capture the configural nature of human shape perception. In order to investigate how the human brain and DCNNs perceive holistic, configural object properties, the research used novel visual stimuli known as "Frankensteins." "Frankensteins are simply objects that have been taken apart and put back together the wrong way around," says Elder. "As a result, they have all the right local features, but in the wrong places."
- North America > United States > Illinois > Cook County > Chicago (0.26)
- North America > Canada (0.06)
Even smartest AI can't match human eye - Gadget
A common artificial intelligence model known as deep convolutional neural networks (DCNNs) does not see objects the way humans do – and that could be dangerous in real-world AI applications. That is the conclusion of Professor James Elder, co-author of a York University study published recently, which finds that AI cannot use something called "configural shape perception", which is standard in human perception for recognising shapes. Published in the Cell Press journal iScience, the paper Deep learning models fail to capture the configural nature of human shape perception is a collaborative study by Elder, who holds the York research chair in human and computer vision and is co-director of York's Centre for AI & Society, co-authored with assistant psychology professor Nicholas Baker at Loyola College in Chicago, a former postdoctoral fellow at York. The study employed novel visual stimuli called "Frankensteins" to explore how the human brain and DCNNs process holistic, configural object properties. "Frankensteins are simply objects that have been taken apart and put back together the wrong way around," says Elder. "As a result, they have all the right local features, but in the wrong places."
- North America > United States > Illinois > Cook County > Chicago (0.26)
- North America > Costa Rica (0.06)
- North America > Canada > Ontario > Toronto (0.06)
- Asia > India (0.06)
AI Not as Efficient as Human Configural Shape Perception
Professor James Elder, who is a co-author of a study published by York University, says that deep convolutional neural networks (DCNNs) do not perceive objects as humans do, with configural shape perception, which could be risky in real-time AI applications. The study was reported in the iScience -- a Cell Press journal. "Deep Learning Models Are Unsuccessful in Capturing the Configural Manner of Human Shape Perception" is a joint study by Elder, a York Research Chair in Human and Computer Vision and a Co-Director of York's Centre for AI & Society, and Nicholas Baker, an Assistant Psychology Professor at Loyola College in Chicago, a former VISTA postdoctoral fellow at York. To discover how the human brain and DCNNs process complete, configural object properties, the scientists used novel visual stimuli known as "Frankensteins." Frankensteins are simply objects that have been taken apart and put back together the wrong way around.
Study highlights how AI models take potentially dangerous 'shortcuts' in solving complex recognition tasks
Deep convolutional neural networks (DCNNs) don't see objects the way humans do--using configural shape perception--and that could be dangerous in real-world AI applications, says Professor James Elder, co-author of a York University study published today. Published in the Cell Press journal iScience, Deep learning models fail to capture the configural nature of human shape perception is a collaborative study by Elder, who holds the York Research Chair in Human and Computer Vision and is Co-Director of York's Centre for AI & Society, and Assistant Psychology Professor Nicholas Baker at Loyola College in Chicago, a former VISTA postdoctoral fellow at York. The study employed novel visual stimuli called "Frankensteins" to explore how the human brain and DCNNs process holistic, configural object properties. "Frankensteins are simply objects that have been taken apart and put back together the wrong way around," says Elder. "As a result, they have all the right local features, but in the wrong places." The investigators found that while the human visual system is confused by Frankensteins, DCNNs are not--revealing an insensitivity to configural object properties.
Even smartest AI models don't match human visual processing: How deep-network models take potentially dangerous 'shortcuts' in solving complex recognition tasks
Published in the Cell Press journal iScience, Deep learning models fail to capture the configural nature of human shape perception is a collaborative study by Elder, who holds the York Research Chair in Human and Computer Vision and is Co-Director of York's Centre for AI & Society, and Assistant Psychology Professor Nicholas Baker at Loyola College in Chicago, a former VISTA postdoctoral fellow at York. The study employed novel visual stimuli called "Frankensteins" to explore how the human brain and DCNNs process holistic, configural object properties. "Frankensteins are simply objects that have been taken apart and put back together the wrong way around," says Elder. "As a result, they have all the right local features, but in the wrong places." The investigators found that while the human visual system is confused by Frankensteins, DCNNs are not -- revealing an insensitivity to configural object properties. "Our results explain why deep AI models fail under certain conditions and point to the need to consider tasks beyond object recognition in order to understand visual processing in the brain," Elder says.